P3b: plane_histograms.csv schema version 3, channels in columns - #23

Follow-up P3b of the normalization plan: plane_histograms.csv is now written in schema version 3, one row per XYC plane (image, Z, T) with every channel in its own CellProfiler-style columns (Intensity_MinIntensity_C0, Histogram_Count_C1_00042, …), using the normalization brick's Milestone 2b writer and PlaneRecord (N41–N44). Plan §12 question 27.

Summary

• pyproject.toml: morphobricks-normalization pinned to commit 0e53e6762617e4297df0593c6977fdfb7b405522 (merge of its Milestone 2b, PR Fix singleton-axis readiness and preserve outputs on failed updates #8) instead of @v0.1.0, with a comment on why (no tags until its v0.2.0).
• planes: iter_xyc_planes / XycPlane yield one table row per XYC plane; its channels are read and binned one at a time (XycPlane.histograms), keeping only the finished histograms. PlaneLayout.channel_count; the memory estimate (whole and per block) adds the finished bin counts of every channel (8 bytes × bins × channels), so one whole/blocks decision covers every channel of every plane. The last decoded page is kept, so an RGB page is decoded once for its three channels. iter_planes is unchanged (conversion and previews still use it).
• metadata, axes, dataset: new error different_channel_counts (N43), before any image is read: in the folder check from the headers (C axis, or RGB samples without one) and in the axis review from the confirmed roles, which supersede the header count (a confirmed missing C axis counts as one channel). Shared builder metadata.channel_count_issues. different_plane_counts now compares Z, T and other planes only. The new code is in CONSISTENCY_CODES, so the notebook's "Dataset consistency" panel shows it; the CLI check and run print it and exit 1.
• run: rows from iter_xyc_planes, sorted by path, z, t; tables of schema version 1 or 2 are recomputed completely (manifest schema_version differs; a v2 table under a v3 manifest is unreadable and also recomputed). Manifest stays version 1 and records schema 3; a file's plane_count counts XYC planes. Pixel warnings check every channel; no_finite_pixels counts planes with an empty channel.
• report, notebook_ui (step 4): read the new layout and pool every channel of a group together exactly as before (report.channel_histograms), so the statistics table and plot are unchanged. A v2 table chosen in step 4 shows the brick's "run the preprocessing brick again" message.
• Docs: contract §2, §5, §8, §9, §11; plan §5, §6, §10, §11 and new question 27; user guide §3, §4, §6, §7, §8; README; AGENTS.md; notebook text (title cell, step 1, "Common problems").

Decisions

• Asked the owner: how a plane's channels are held in memory → one channel at a time (recommended option), not all channels together. Block decisions therefore stay as before except for the small histogram term.
• Step 4 keeps pooling all channels of a group together, because the spec says its table and plot must not change; the "planes" column keeps counting channel planes. Per-channel statistics belong to P5.
• Deprecated re-exports (_moved.py) keep forwarding the moved names to the brick's current objects (now with Milestone 2b signatures, e.g. bin_column(channel, index)). Names the brick added later (channel_column, WeightedHistogram, different_channel_counts, …) are not forwarded; test_moved now checks the moved names are a subset of the brick's.
• Experiment design, OME-Zarr copies and the manifest version are unchanged. PR Input support for MorphoRelate M3: single folders, physical sizes per axis (TIFF and OME-Zarr), embeddable notebook steps #20's decisions (questions 23–26) are kept.

Notes for review

• Several existing tests used a missing C axis next to multi-channel images as the example of "missing axes"; under N43 that is now different_channel_counts, so they use a missing T axis instead, and new tests cover the missing-C case.
• When C is a page axis and pages hold several Z/T planes (volumetric TIFF with C outside the page), a page is decoded again for each XYC plane: time, never memory. Documented in planes.
• The step-1 header check can report different_channel_counts together with missing_axes / inconsistent_dimensions (e.g. ZCYX next to ZYX); after the axis roles are confirmed, the review's own count replaces it.
• Not verified: the notebook in a real JupyterLab (widgets tested headless only; no rendered visual check this time, the UI changes are text and an existing error style); Python 3.11 locally (no interpreter here; CI covers it). Zarr 2.18 was checked locally on Python 3.12 for the zarr, run, planes and equivalence tests.

Tests

• New tests/test_schema_v2_equivalence.py with a committed fixture in tests/data written by the pre-P3b code (55f1074 + normalization v0.1.0, via tests/data/make_v2_fixture.py): uint16 (TIFF plain, tiled-compressed, ImageJ, and an OME-Zarr image in another axis order; 500 bins, exact and binned channels) and float32 (NaN/±inf, an all-NaN channel, a constant channel; 64 bins). Every channel of every v3 row has the same text as the v2 row for that channel, read whole and in blocks; step 4 statistics equal those pooled from the v2 rows; a v2 table is recomputed on the next run.
• New tests for XYC reading on every axis layout and every block layout, the memory estimate per channel, the channel-count error (headers, confirmed roles, CLI, notebook panel).
• Windows, Python 3.12, zarr 3.4.1, .[dev,notebook,zarr]: ruff check . and ruff format --check . clean; 935 passed, 2 skipped.
• Same with zarr 2.18.7: zarr/run/planes/equivalence tests 417 passed, 16 skipped (OME-NGFF 0.5).

🤖 Generated with Claude Code